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ABSTRACT 

Synthetic Vision Systems and Enhanced Flight Vision System (SVS/EFVS) technologies have the potential to provide 
additional margins of safety for aircrew performance and enable operational improvements for low visibility operations 
in the terminal area environment. Simulation and flight tests were jointly sponsored by NASA’s Aviation Safety 
Program, Vehicle Systems Safety Technology project and the Federal Aviation Administration (FA A) to evaluate 
potential safety and operational benefits of SVS/EFVS technologies in low visibility Next Generation Air Transportation 
System (NextGen) operations. The flight tests were conducted by a team of Honeywell, Gulfstream Aerospace 
Corporation and NASA personnel with the goal of obtaining pilot-in-the-loop test data for flight validation, verification, 
and demonstration of selected SVS/EFVS operational and system-level performance capabilities. 

Nine test flights were flown in Gulfstream’ s G450 flight test aircraft outfitted with the SVS/EFVS technologies under 
low visibility instrument meteorological conditions. Evaluation pilots flew 108 approaches in low visibility weather 
conditions (600 feet to 3600 feet reported visibility) under different obscurants (mist, fog, drizzle fog, frozen fog) and 
sky cover (broken, overcast). 

Flight test videos were evaluated at three different altitudes (decision altitude, 1 00 feet radar altitude, and touchdown) to 
determine the visual advantage afforded to the pilot using the EFVS/Forward-Fooking InfraRed (FFIR) imagery 
compared to natural vision. Results indicate the EFVS provided a visual advantage of two to three times over that of the 
out-the-window (OTW) view. The EFVS allowed pilots to view the runway environment, specifically runway lights, 
before they would be able to OTW with natural vision. 
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1. INTRODUCTION 

The U.S. air transportation system is undergoing a transformation to accommodate the movement of large numbers of 
people and goods in a safe, efficient, and reliable manner. 1 One of the key capabilities envisioned to achieve this Next 
Generation Air Transportation System (NextGen) is the concept of equivalent visual operations (EVO). EVO is the 
capability to achieve the safety of current-day Visual Flight Rules (VFR) operations and maintain the operational tempos 
of VFR irrespective of the weather and visibility conditions. 

One research challenge for EVO is the definition of required equipage on the aircraft and at the airport. With today’s 
equipment and regulations, significant investment is required in on-board equipment for navigation, surveillance, and 
flight control and on the airport for precision guidance systems and approach lighting systems for “all-weather” landing 
capability. 2 The levels of equipment redundancy, capability, maintenance, performance, and crew training dramatically 
increase as landing visibility minima decrease. Synthetic Vision Systems and Enhanced Flight Vision Systems 
(SVS/EFVS) offer a means of providing EVO capability with significant economic advantages to the operator yet 
without significant airport infrastructure investment while potentially increasing efficiency and throughput during low 
visibility operations. 

SVS is a computer-generated image of the external scene topography, generated using aircraft attitude, high-precision 
navigation, and data of the terrain, obstacles, cultural features, and other required flight information. This computer 
rendered view of the outside world may be presented on a head-down display or head-up display (HUD). EFVS is an 
electronic means to provide a display (typically on a HUD) of the forward external scene by use of an imaging sensor, 
such as a Forward-Fooking InfraRed (FFIR) or millimeter wave radar. Both SVS and EFVS are “vision-based” 
technologies intended to create, supplement, or enhance the natural vision of the pilot. 
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1.1 Current EFVS rules/operations 

In 2004, Chapter 14 of the Code of Federal Regulations (CFR) Section §91.175 was amended such that operators with 
approved EFVS equipment that are conducting straight- in instrument approach procedures may now operate below the 
published Decision Altitude (DA), Decision Height (DH), or Minimum Descent Altitude (MDA) when using an 
approved EFVS shown on the pilot’s HUD. The key concept under the revisions to §91.175 is that an EFVS can be used 
in lieu of the required natural vision from the DA/DH/MDA to 100 feet (ft) height above the touchdown (HAT) zone 
elevation. At the 1 00 ft HAT, pilots must have sufficient natural vision as required by the instrument approach procedure 
and to identify the required landing visual references to continue the approach. In addition, FAA Advisory Circular AC 
20-167 3 provides guidance on certification and installation of EFVS, and FAA Advisory Circular AC 90-106 4 provides 
guidance for obtaining operational approval to use EFVS in lieu of natural vision to descend below DA/DH or MDA. 

1.2 Proposed EFVS operations changes 

The FAA started a rulemaking project to expand operational credit for EFVS beyond what is currently authorized by the 
regulations (under 14 CFR § 91.175). 5,6 RTCA DO-315A 7 was drafted to establish performance standards in concert with 
this rulemaking project. Minimum system performance standards are now published for EFVS operations through the 
approach to touchdown in visibility as low as 1000 ft runway visual range (RVR) by use of an approved EFVS in lieu of 
natural vision. Simply stated, (in DO-3 15 A 7 ) the visual segment of the approach can now be accomplished by using 
either enhanced flight visibility or natural vision. Enhanced flight visibility is defined in the Federal Aviation 
Regulations (FARs) as “the average forward horizontal distance, from the cockpit of an aircraft in flight, at which 
prominent topographical objects may be clearly distinguished and identified by day or night by a pilot using an enhanced 
flight vision system.” Past NASA research 8 supports the viability of this expanded EFVS visual segment where it was 
shown that using an EFVS to hand fly approaches to touchdown resulted in excellent localizer tracking performance 
(less than 1/3 dot localizer deviation between 300 ft and 100 ft HAT) and an improvement in glideslope tracking 
performance. 

On June 11, 2013, the FAA published a Notice of Proposed Rulemaking (NPRM), FAA-2013-0485 Notice Number 
1209, to permit operators to use an EFVS in lieu of natural vision to continue descending from 100 ft HAT through 
touchdown and rollout on certain straight- in instrument approaches in visibilities as low as 1000 RVR. 9 The rule would 
also permit certain EFVS operators to dispatch, release, or takeoff under instrument flight rules and initiate and continue 
an approach, when the destination airport weather is below authorized visibility minimums for the runway of intended 
landing. The NPRM includes pilot training, flight experience, and proficiency requirements for operators who use EFVS 
in lieu of natural vision to descend below DA, DH or MDA and revisions for the pilot compartment view certification 
requirements for all vision systems. 

1.3 SVS/EFVS flight test 

Simulation 10 and flight test 11 experiments were conducted as part of a formal collaboration between NASA Langley 
Research Center (LaRC) and the FAA under an Interagency Agreement (I A) to support the introduction and use of 
SVS/EFVS advanced cockpit vision technologies in NextGen operations. This work builds from and extends the current 
operational use and certification of existing SVS/EFVS technologies toward all-weather, low visibility operations for 
NextGen. 

Of particular note, an SVS/EFVS flight test 11 was conducted, in partnership with Honeywell and Gulfstream, to evaluate: 

1 . Operational feasibility, pilot workload, and pilot acceptability of conducting a straight-in instrument approach 
with published vertical guidance using EFVS during approach, landing, roll-out, and runway exit in visibility of 
1000 ft RVR. 

2. Operational feasibility, pilot workload, and pilot acceptability of conducting an instrument landing system 
approach to a 150 ft Decision Height (DH) using SVS followed by a transition to natural out-the-window 
(OTW) visual cues for landing with the visibility as low as 1400 ft RVR. 

The focus of this paper is to attempt to quantify the visual advantage (or increase in forward visual range) afforded to a 
pilot using an imaging sensor (EFVS/FLIR) compared to natural vision while making approaches in low visibility 
operating conditions with varying weather obscurants and sky cover. As such, only the EFVS flight trials were 
considered. 



Flight test video from nine flights flown in Gulfstream’s G450 flight test aircraft outfitted with Kollsman EFVS II 
technology under low visibility instrument meteorological conditions were used to determine the visual advantage 
benefits. During this flight test, evaluation pilots flew 108 approaches in low visibility weather conditions (600 ft to 
3600 ft reported visibility) under different obscurants (mist, fog, drizzle fog, frozen fog) and sky cover (broken, 
overcast). Flight test video were evaluated at three different altitudes (decision altitude, 100 ft radar altitude, and 
touchdown) to assess if a ‘visual advantage’ can be quantified using the EFVS/FLIR imagery compared to natural 
vision. Details of the test set-up and operation are contained in Reference 1 1 . 

2. METHODOLOGY 


2.1 Test aircraft 

The flight test was conducted using Gulfstream’s G450 flight test aircraft N401SR, S/N 4001 (Figure 1). The test aircraft 
was equipped with certified avionics and software, including the Honeywell Synthetic Vision-Primary Flight Display 
(SV-PFD) and monochromatic EFVS Head-Up Display (HUD) with display of conformal symbolic information, flight 
information, and FLIR imagery (Figure 2). The HUD is the Rockwell-Collins’ model HGS 6250, and the FLIR is the 
Kollsman Enhanced Vision System (EVS) II infra-red camera. 



Figure 1. Gulfstream G450 test aircraft. 


The G450 is one of the Gulfstream test aircraft used for on-going test and certification activities and is equipped to 
record data and video. The G450 test aircraft’s avionics were not experimental equipment but were the current standard 
avionics suite as certified and in service. The G450 aircraft was also certified with an EFVS system which allows 
“operational approval” for approaches with descent below published minima down to 100 ft HAT as documented in 
Code of Federal Regulations (CFR) 14 Part 91.175(1) and discussed in Advisory Circular AC 90- 106. 4 



Over 300 parameters were recorded at 50 samples per second during each approach and landing run. Several relevant 
parameters, including path error (localizer, glideslope, vertical speed) and touchdown performance (sink rate and speed 
at touchdown, distance from the threshold, and distance left or right of centerline), were measured for analysis. Nine 
channels of video were recorded on three separate digital video recorders (DVR). The three DVR recordings were 
comprised of a quad arrangement of cameras (Figure 3, left picture), a quad arrangement of head-down instruments 
(Figure 3, right picture), and) and a single channel of EFVS Imagery (FLIR) and Symbology (Figure 2, right picture). 


Quad Camera Recording 



Figure 3. Quad camera recording (left picture) and quad instrument recording (right picture). 



Quad Instrument Recording 


The nine video sources included: 


• EVS Camera (Raw FLIR imagery) 

• HUD Camera 

• Visual Out the Window Camera 

• Cockpit Area Camera 

• Left Seat Primary Flight Display 


Left Seat Navigation/Multi-Purpose Display 
Right Seat Navigation/Multi-Purpose Display 
Right Seat Primary Flight Display 
Combined HUD & EFVS Imagery presented 
on the HUD 









2.2 Evaluation pilots 

Six Evaluation Pilots (EPs) participated in the flight evaluations and represented a diverse mix of experience. 

The EPs met the following experience criteria: 

• Each pilot held an Airline Transport Pilot rating 

• Each pilot had significant HUD experience, having flown at least 100 hrs of HUD, pilot-in-command 
operations. 

• Each pilot was type-rated in a Gulfstream G-IV, G-V, and G450, G550, or G650 aircraft and had EFVS 
qualifications and EFVS operational experience. Total time in the Gulfstream aircraft was greater than 200 hrs. 

The average experience of the EPs was 9100 hours of flight time with an average of 28 years of flying. Most of the EPs 
flew on multiple evaluation flights. 

2.3 Procedures 

All EFVS testing was flown with the EP occupying the left seat using the HUD, a Gulfstream Safety Pilot (SP) 
occupying the right seat, and an additional Safety Observer (SO) occupying the center jump seat. EFVS test operations 
were conducted under an FAA waiver to the current Title 14 of the CFR §91.175 allowing these test flights (i.e., in an 
operation that is not currently approved). This waiver allowed the use of EFVS or natural vision to see the required 
visual references, as to continue descent below the DA/DH through landing. Safety procedures required that the SP have 
positive visual acquisition of the required landing references by 50 ft above touchdown elevation. 

The EPs flew straight-in instrument approaches adhering to published approach procedures (other than the waivered 
minimums) to the runway. All approaches were flown with the EP manually flying the aircraft below 1000 ft above 
ground level to a landing. The initial approach procedure was often flown with the auto-pilot engaged, following the 
approach procedure. Auto-throttles were used for all approaches. Detailed call-out procedures for both the EP and SP 
were utilized for all approaches ensuring safety throughout the approach and providing clear evidence of the acquisition 
of the required approach and landing visual references as per §91.175. 

2.4 Test operations 

All test flights originated at the Savannah/Hilton Head International Airport (KSAV) which is the location of the 
Gulfstream manufacturing and flight test facilities. Table 1 lists all of the airports and runways utilized in the flight test 
operations. 

The test locations were determined in near real-time using weather operations support and from Automated Surface 
Observing System (ASOS) and Automated Terminal Information Services (ATIS) reports while in route to the target 
area to locate low visibility conditions. The actual operating conditions obviously changed from these targets until the 
aircraft arrived on station, and they varied as approach and landing operations were conducted. 

2.5 Test conditions - weather 

For EFVS operations the target visibility was 1000 ft RVR to 1/4 statute mile (SM) with a ceiling of 100 feet. The exact 
desired weather conditions for the test were achieved on several occasions. Over the course of multiple approaches, the 
visibility and ceiling naturally varied at each airport, giving a nice range of data on either side of the target conditions. A 
sampling of weather conditions for data collection at the different airports is shown in Table 1. 



Table 1. Test airports and conditions. 


Airport 

Approach 

Reported 

Visibility 

(SM) 

Reported 

RVR 

(ft) 

Reported 

Ceiling 

(«) 

Weather Notes 

Shenandoah Valley 
Regional 

KSHD ILS 05 

1/4- 1/2 


100 

Fog, Overcast, Thin but dense 
layer 

Altoona Blair County 

KAOO ILS 21 

1/2 


100 

Fog, Overcast 

Portsmouth Intl./Pease 

KPSM ILS 34 

1/8 

900-1200 

100 

Fog, Broken 

Bar Harbor -Hancock 
County 

KBHB ILS 22 

1/4 


200 

Fog, Overcast 

Greater Binghamton/ 
Edwin A. Link Field 

KBGM ILS 16 

< 1/4 

600 

100 

Fog, Indefinite ceiling 

Albany International 

KALB ILS 01 

1/4 


200 

Mist 

Vidalia Regional 

KVDI ILS 24 

1/4 


100 

Fog, Overcast 

Savannah/Hilton Head 
International 

KSAV ILS 10 

1 1/4 


200 

Scattered thin 

Acadiana Regional 

KARA ILS 34 

1/4 


100 

Broken, Overcast 

Dense thin layer at 100-50 ft 

Esler Regional 

KESF ILS 27 

1/4 


100 

Fog, Overcast 

Alexandria 

International 

KAEX ILS 14 

1/4 


100 

Fog, Overcast 

Cincinnati Muni 
Airport-Lunken Field 

KLUK ILS 21 

1/4 

2400 

100 

Fog 

Akron- Canton 
Regional 

KCAKILS 19 

1/4 


100 

Freezing Fog 

Y oungstown/W arren 
Regional 

KYNG ILS 32 

1/8 

1000 

200 

Freezing Fog 


2.6 Flights summary 

There were seven data collection flights conducted from July 20 to Oct 28, 2011. All flight time totaled approximately 
38 hours with the seven data collection flights comprising approximately 35 flight hours. Data were collected at 15 
different airports and utilized 16 different runways. 

There were 107 approaches flown, 80 were EFVS approaches (75%) and 27 were SVS approaches (25%). Only the 
EFVS approaches are applicable to the evaluation of visual advantage which is the focus of this paper. Out of the 80 
EFVS approaches, seven were culled out of the data analysis for various extraneous reasons such as: Approach 
Lightning System (ALS) automatically turning off, or EP mistakenly left autopilot on during much of the approach, etc. 
These events were anomalous and caused significant deviations from our nominal operation and therefore, were not 
representative of the other approaches. 

Of the 73 useable EFVS approach evaluations, 53 (73%) resulted in a touchdown and 20 (27%) resulted in missed 
approach. Eight of the EFVS approaches were to an offset runway (KBGM). The 20 missed EFVS approaches were all 
conducted safely with the go-around decision correctly determined based on conditions. Both touchdown and missed 
approach runs were analyzed in this paper. 




2.7 Determination of forward horizontal visible distance from cockpit 


Snapshots of flight test video were taken from the quad camera DVR recordings (Figure 3, left picture) and EFVS DVR 
recordings (Figure 2, right picture) at the Decision Altitude (DA), 100 feet radar altitude (RA) call out, and as the plane 
touched down. The 100 ft RA audio callout was used as an approximation of 100 ft HAT due to difficulties in 
determining 100 ft HAT using the available video playback data. The 100 ft RA audio callout was available on each 
DVR recording so consistent snapshots could be made among the recordings. The quad camera recording was comprised 
of the EVS (FLIR) camera, visual (OTW) camera, HUD camera, and a cockpit camera. The snapshots from the quad 
camera DVR recordings were used to count the lights that approximated what the pilot could see using the FLIR camera 
and OTW and thus, attempt to quantify the EFVS visual advantage. The snapshots from the EFVS DVR recordings were 
used to estimate the distance to the first visible light at DA and 100 ft RA. 


5 th light 
past threshold 

Threshold lights 


2 nd light 
past threshold 

Threshold lights 



Figure 4. Quad camera recording. 

Visual advantage was estimated by counting the runway edge lights that were visible in the respective video images, 
starting with the threshold lights. The threshold lights were counted as one light if they were visible. Each runway edge 
light was counted as one light if visible. For example, in the quad camera image seen in Figure 4, in the EVS view 
(upper left camera), the threshold lights are visible in addition to 5 runway edge lights on the left side of the runway. 
This would count as 6 lights visible through the EVS. Two runway edge line lights plus the threshold lights are visible 
out the window on the left side of the runway (Figure 4, bottom left camera). This would count as three lights visible 
OTW, compared to the six seen through the EVS. The runway edge lights are 200 feet apart. The touchdown lights and 
visual approach slope indicator (VASI) lights were used if needed to order to estimate runway edge lights that were 
further down the runway. The VASI lights are normally 1000 feet beyond the runway threshold. The touchdown zone 
lights, if equipped, extend 3,000 feet down the runway and are placed at 100 foot increments. 

In most cases, at the DA, the MALSR (Medium-intensity Approach Lighting System with Runway Alignment Indicator 
Lights) approach lighting system produced the only visible lights. MALSRs consist of nine light bars and five flashing 
lights spaced every 200 feet as you approach the runway. These lights, in addition to any visible runway edge lights, 
were counted and recorded. 

The altitude readout in the HUD camera image (Figure 3, left picture upper right quadrant) was used to determine when 
the aircraft was crossing the DA, and the radar altimeter was used to determine when the aircraft had touched down. The 
HUD camera was not synchronized with the HUD projector, and this occasionally caused the HUD camera image to 
fade in and out. (The HUD projector image which was presented to the EP during EFVS runs was steady and did not 
fade in and out.) Snapshots were not taken for runs where the HUD camera image was unavailable at the DA or 
touchdown. As a result, there were not an equivalent number of snapshots taken at the DA, 1 00 ft RA, and touchdown 
points due to this synchronization issue. 


The number of visible lights was used to calculate the forward horizontal distance from the cockpit that was visible to 
the pilot through the EFVS and OTW. Since all counted lights were recorded as being 200 feet apart, the number of 
lights was multiplied by 200 and recorded as being the forward horizontal visible distance. This procedure assumed that 
the aircraft was 200 feet from the first light. 

Since this assumption isn’t always the case, the distance to the first light was estimated. The look-down angle from the 
horizon line to the first visible light (referred to as ^ in the figure below) was estimated using the pitch ladder from the 
snapshots taken from the EFVS DVR recordings (Figure 2, right picture). This measurement was done at the DA and at 
100 feet RA. Overall, this provided a better estimate of how far down the runway the pilot could see. 



Figure 5. Visible distance to first visible light. 

2.8 Determination of reported visibility, obstruction, and sky cover 

If the reported RVR was available, then RVR was used as reported visibility for this analysis. If reported visibility in 
statute miles was the only visibility available then it was converted from miles to feet and recorded for reported 
visibility. It should be remembered that reported visibility does not necessarily represent the actual visibility that the 
pilot might experience coming into an approach. 

Meteorological Terminal Aviation Routine Weather Report (METAR) data was also recorded for each approach 
performed during the flight test. A METAR historical record ( http://www.ogimet.com/metars.phtml.en) and METAR 
translator ( http://www.iflightplanner.com/Resources/MetarTafrranslator.aspx) were used to record this data. The 
reported obstruction and sky cover were provided in the METAR data. Obstructions that were reported during the EFVS 
flight test runs included fog, mist, drizzle fog, and frozen fog. Reported sky cover included overcast and broken sky 
cover. 


3. RESULTS 

Visual advantage is defined as an increase in average forward horizontal distance from the cockpit of an aircraft 
provided by an imaging sensor, such as FLIR, over that provided by natural vision. 

Two metrics, the visual advantage difference and the visual advantage factor, were used to quantify any increase in 
forward horizontal visible distance by EFVS over natural vision. The visual advantage difference was calculated by 
subtracting the mean EFVS visual distance by the mean OTW visible distance. A positive value of this metric indicates 
visual advantage by the EFVS (FLIR sensor) over natural vision. The visual advantage factor was calculated by dividing 
the mean distance visible through the EFVS by the mean distance visible OTW. Note that if there were no lights visible 
OTW, then the visual advantage factor could not be calculated. 

The forward horizontal visible distance was quantified by calculating the horizontal visual distance from the pilot’s eye 
to the first light, using the look down angle, and adding to this the number of counted visible runway edge lights, 
excluding the first visible light, and multiplying by 200, the distance between runway edge lights. The weighted average 
of the visual distance was calculated at the decision altitude, 1 00 ft radar altitude, and touchdown. The weighted average 
takes into consideration the number of approaches used to calculate the visual distance. 


3.1 Visual advantage by reported visibility at 100 ft radar altitude 

Table 2 provides the visual advantage difference, the visual advantage factor (if calculable) and the number of 
approaches flown at each reported visibility level at three distinct points on the approach (DA, 100 ft RA, and 
touchdown). Also provided in this table are the weighted averages for the two visual advantage metrics at the 3 distinct 
approach points. 


Table 2. Visual advantage under reported visibilities at distinct points of approach. 


Approach 

Point 

Reported 
Visibility (ft) 

EFVS 

Distance (ft) 

OTW 

distance 

(ft) 

Visual 
Advantage 
Difference (ft) 

Visual 

Advantage 

Factor 

Number of 
Approaches 

DA 

600 

2525 

1641 

884 

1.5 

4 

900 

2087 

0 

2087 

- 

3 

1000 

- 

- 

- 

- 

- 

1200 

2295 

961 

1334 

2.4 

24 

1800 

2109 

0 

2109 

- 

1 

2400 

1992 

0 

1992 

- 

2 

3000 

- 

- 

- 

- 

- 

3600 

2752 

1552 

1200 

1.8 

1 

Weighted Average 

2294 

891 

1403 

1.9 


100 ft RA 

600 

619 

154 

465 

4.0 

7 

900 

1744 

901 

843 

1.9 

7 

1000 

484 

0 

484 

- 

1 

1200 

1530 

654 

876 

2.3 

37 

1800 

1886 

0 

1886 

- 

1 

2400 

1694 

379 

1315 

4.5 

4 

3000 

1098 

1198 

-100 

0.9 

1 

3600 

1055 

322 

733 

3.3 

2 

Weighted Average 

1425 

582 

843 

2.6 


TD 

600 

733 

33 

700 

22.0 

3 

900 

1357 

871 

486 

1.6 

7 

1000 

700 

0 

700 

- 

1 

1200 

1335 

111 

618 

1.9 

33 

1800 

1100 

200 

900 

5.5 

1 

2400 

1300 

350 

950 

3.7 

4 

3000 

1500 

1400 

100 

1.1 

1 

3600 

1100 

450 

650 

2.4 

2 

Weighted Average 

1278 

649 

629 

3.2 



The average forward horizontal visible distance provided by the EFVS was greater than that provided by natural vision 
at the three distinct approach points for all reported visibility levels, except for the 3000 ft visibility level at the 100 ft 
RA approach point (Table 2 and Figure 6). For that one approach, the pilot could see approximately 1098 feet through 
the EFVS and 1198 feet OTW. These visual distance values are considered to be essentially equivalent. 


With weighted average visual advantage factor values of 1.9 to 3.2 (Table 2), during these approaches the EFVS 
provided forward horizontal visible distance increases of two to three times that of the OTW natural vision view. The 
EFVS allowed pilots to view the runway environment, specifically runway lights, before they would be able to out the 
window. 
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Figure 6. Forward horizontal visible distance at reported visibilities at 100 ft RA. 


Examination of boxplots at the DA, 100 ft RA, and touchdown (Figures 7-9, respectively) show that the data is skewed 
for the OTW forward horizontal visible distance. On each box, the central mark is the median, the edges of the box are 
the 25th and 75th percentiles, the whiskers extend to the most extreme data points not considered outliers, and outliers 
are plotted individually. The pilots did not see any lights with natural vision (i.e., zero ft visible distance OTW) for 57% 
of the approaches at DA, 40% of the approaches at 100 ft RA, and 21% of the landings, which resulted in lowering the 
mean OTW visible distance values (Table 2) at the applicable reported visibility levels. Table 3 shows the mean forward 
horizontal visible distances for the EFVS and natural vision and the visual advantage factor at the three approach points 
when using only the OTW approaches with non-zero values. The EFVS still provided increases in forward horizontal 
visible distance 1.2 to 1.7 times greater than that provided by natural vision when considering only the non-zero OTW 
visible distance values. 

Even though the reported visibility was between 600 and 3600 ft for the approaches flown, the estimated OTW forward 
visible distance varied greatly at the three approach points where it ranged from a minimum of zero ft to maximum 
values of 5510 ft at DA, 3014 at 100 ft RA, and 1650 ft at touchdown (Figures 7-9). The EFVS forward visible distance 
ranged from 1060-5710 ft at DA, 221-3610 ft at 100 ft RA, and 600-1750 at touchdown. 


Table 3. Visual advantage factor when considering only approaches with non-zero OTW visible distance values. 


Approach Point 

EFVS Distance (ft) 

OTW Distance (ft) 

Visual Advantage Factor 

Number of Approaches 

DA 

2545 

2078 

1.2 

15 

100 ft RA 

1734 

970 

1.8 

36 

TD 

1377 

823 

1.7 

41 


Forward Horizontal Visible Distance (ft) R Forward Horizontal Visible Distance (ft) 


DA 



7. Box plot of forward horizontal visible distance at reported visibilities at Decision Altitude (DA). 
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Figure 8. Box plot of forward horizontal visible distance at reported visibilities at 100 ft RA. 
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Figure 9. Box plot of forward horizontal visible distance at reported visibilities at touchdown. 


3.2 Visual advantage by obscurant and approach point 

Table 4 provides the visual advantage metrics and number of approaches flown within each obscurant type at three 
distinct points on the approach (DA, 100 ft RA, and touchdown). Table 4 also provides the weighted averages for the 
two visual advantage metrics at the three distinct approach points. 


Table 4. Visual advantage under different obscurants at distinct points of approach. 


Approach 

Point 

Obscurant 

EFVS 

Distance 

(«) 

OTW 

Distance 

(ft) 

Visual 
Advantage 
Difference (ft) 

Visual 

Advantage 

Factor 

Number of 
Approaches 

DA 

Frozen Fog 

3096 

0 

3096 

- 

2 

Drizzle Fog 

2494 

779 

1715 

3.2 

6 

Fog 

2174 

1134 

1039 

1.9 

22 

Mist 

2263 

310 

1953 

7.3 

5 

Weighted Average 

2294.0 

890.7 

1403.3 

2.8 


100 ft RA 

Frozen Fog 

1701 

754 

947 

2.3 

4 

Drizzle Fog 

2327 

744 

1583 

3.1 

6 

Fog 

1202 

549 

653 

2.2 

39 

Mist 

1623 

549 

1074 

3.0 

11 

Weighted Average 

1424.9 

582.2 

842.7 

2.4 


TD 

Frozen Fog 

1150 

600 

550 

1.9 

4 

Drizzle Fog 

1550 

1080 

470 

1.4 

5 

Fog 

1247 

605 

642 

2.1 

32 

Mist 

1291 

600 

691 

2.2 

11 

Weighted Average 

1277.9 

649.0 

628.8 

2.0 

1277.9 


Under all reported obscurants, including mist, fog, drizzle fog, and frozen fog, the forward horizontal visible distance 
provided by the EFVS was greater than that provided by natural vision at the decision altitude, 100 feet RA, and 
touchdown of each flight. When flying in frozen fog, the pilot did not see any runway lights using natural vision at the 
decision altitude, but the pilot was able to see an average of 3096 feet forward horizontal distance from the cockpit using 
EFVS (Figure 10). This distance is a significant improvement in visual range. Across the combinations of approach point 
and obscurant (Table 4), the visual advantage difference metric ranged from 470 to 3096 ft and the visual advantage 
factor metric indicated that the EFVS provided increases in forward horizontal visible distance 1.4 to 7.3 times greater 
than that provided by natural vision. With weighted average visual advantage factor values of 2.0 to 2.8 (Table 4), the 
EFVS provided forward horizontal visible distance increases of two to three times that of the OTW natural vision view 
when flying through various types of fog and mist. The EFVS allowed pilots to view the runway environment, 
specifically runway lights, before they would be able to out the window. This increases a pilot’s situational awareness 
and provides for safer landings. 
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Figure 10. Forward horizontal visible distance under different obscurants at distinct approach points. 


As noted in Section 3.1, the mean OTW forward horizontal visible distances (Table 4) are skewed due to approaches 
where the value for this measure was zero ft (i.e., no lights visible OTW). For the obscurant mist, 80% of the DA 
approaches and 37% of the 100 ft RA approaches had no lights visible OTW in the snapshots taken at these approach 
points, but the mean EFVS visible distances for these approaches were 2141 ft at DA and 1378 ft at 100 ft RA. 
Similarly, for the obscurant fog, 50% of the DA approaches, 46% of the 100 ft RA, and 31% of the landings had no 
lights visible OTW in the snapshots taken at these approach points, but the mean EFVS visible distances for these 


approaches were 1271 ft at DA, 926 ft at 100 ft RA, and 920 ft at landing. The EFVS effectively increased the pilot’s 
forward horizontal visible distance by 1000 ft. 

3.3 Visual advantage by sky cover and approach point 

Table 5 provides the visual advantage metrics and number of approaches flown within each sky cover type at three 
distinct points on the approach (DA, 100 ft RA, and touchdown). Table 5 also provides the weighted averages for the 
two visual advantage metrics at the three distinct approach points. 


Table 5. Visual advantage under different sky cover at distinct points of approach. 


Approach 

Point 

Sky Cover 

EFVS 

Distance 

(fo 

OTW 

Distance 

(«) 

Visual 
Advantage 
Difference (ft) 

Visual 

Advantage 

Factor 

Number of 
Approaches 

DA 

Overcast 

2100 

616 

1484 

3.4 

30 

Broken 

3460 

2540 

920 

1.4 

5 

Weighted Average 

2294.0 

890.7 

1403.3 

3.1 


100 ft RA 

Overcast 

1412 

485 

927 

2.9 

54 

Broken 

1543 

1460 

83 

1.1 

6 

Weighted Average 

1424.9 

552.2 

842.7 

2.7 


TD 

Overcast 

1267 

594 

673 

2.1 

47 

Broken 

1380 

1170 

210 

1.2 

5 

Weighted Average 

1277.9 

649.0 

628.8 

2.0 



Two different types of sky cover were reported during the flight test, overcast and broken. The majority of approaches 
(89%) were flown under overcast conditions. Under these two sky covers, the average forward horizontal visible 
distance provided by the EFVS was greater than that provided by natural vision at the DA, 100 ft RA, and TD approach 
points (Figure 11). On average, the EFVS provided visual distances 1.2 to 2.8 times greater than natural vision when 
flown in broken and overcast conditions, respectively. The visual advantage difference ranged from 83 to 1484 feet with 
weighted averages between 629 and 1403 ft. The visual advantage factor ranged from 1.1 to 3.4, with weighted averages 
between 2.0 and 3.1 (see Table 5). Again, both metrics indicate that an increase in forward horizontal visible range was 
provided by the EFVS compared to natural vision under two different types of sky cover. 
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Figure 11. Forward horizontal visible distance under different sky cover at distinct approach points. 

3.4 Visual advantage by ceiling and approach point 


The ceiling during each flight of the flight test was reported as 100 feet, 200 feet, 300 feet, or 500 feet. The forward 
horizontal visible distance from the cockpit with the EFVS and natural vision was calculated at DA, 100 ft RA, and TD, 
under these four different ceilings (Table 6 and Figure 12). Under the ceiling levels flown, the EFVS provided a visible 
range increase at least equivalent to or greater than that provided OTW. The visual advantage difference metric ranged 
from 600 to 2109 ft, with weighted averages ranging from 629 to 1403. Across the combinations of approach point and 
ceiling, the visual advantage factor ranged from 1.8 to 5.5, with weighted averages ranging from 2.1 to 2.5, as seen in 
Table 6. Under the ceiling levels flown, pilots had more than twice the forward horizontal distance from the cockpit (i.e., 
a much better visual image of the runway environment) when using the EFVS than when using natural vision. 


Of the runs evaluated in this paper, 80% of the DA approaches, 61% of the 100 ft RA approaches, and 69% of the 
landings were flown under a 100 ft ceiling. For these approaches, the EFVS provided approximately two times more 
forward visible distance than that provided by natural vision which mirrors the results found with the weighted visual 
advantage factors at the three approach points. 


Table 6. Visual advantage under different ceilings at distinct points of approach. 


Approach 

Point 

Ceiling (ft) 

EFVS 

Distance 

(ft) 

OTW 

Distance 

(«) 

Visual 
Advantage 
Difference (ft) 

Visual 

Advantage 

Factor 

Number of 
Approaches 

DA 

500 

2109 

0 

2109 

- 

1 

300 

- 

- 

- 

- 

0 

200 

2494 

779 

1715 

3.2 

6 

100 

2258 

946 

1311 

2.4 

28 

Weighted Average 

2294.0 

890.7 

1403.3 

2.5 


100 ft RA 

500 

1886 

0 

1886 

- 

1 

300 

966 

0 

966 

- 

1 

200 

1536 

456 

1080 

3.4 

15 

100 

1386 

653 

733 

2.1 

43 

Weighted Average 

1424.9 

582.2 

842.7 

2.4 


TD 

500 

1100 

200 

900 

5.5 

1 

300 

1200 

600 

600 

2.0 

1 

200 

mi 

450 

661 

2.5 

14 

100 

1350 

740 

610 

1.8 

36 

Weighted Average 

1277.9 

649.0 

628.8 

2.1 
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Figure 12. Forward horizontal visible distance under different ceilings at distinct approach points. 


3.5 Caveats 


The results generally show a visual advantage of EFVS over natural vision. These results correlate well with the 
operational advantage identified by the evaluation pilots in the flight test. 11 This advantage was quantified using post- 
flight video analysis. In doing this analysis, several caveats or considerations must be considered in the universal 
applicability or general validity of the method. Significant differences between this post- flight analysis method and real- 
time flight operations include the factors: 

a) the resolution and dynamic range of the video cameras and recording devices used to capture the data; 

b) the monitors to view the recorded data; 

c) the imaging differences between the EFVS raw data video used in the analysis and the EFVS video as shown on 
the HUD for the EP; 

d) the imaging differences between the raw data OTW video used in our analysis and the pilot’s view through the 
windows for the EP 

e) Field of view, position, and alignment of the respective cameras 

By design of the G450 test vehicle, the EFVS cameras and OTW video camera had very similar field-of-view, 
positioning, and alignment so good comparison was afforded for this factor. Calibration methods would be necessary to 
verify the other factors; these were not conducted. 

The most dramatic differences between the methods, however, is intrinsic: a post- flight, full attention, frame-by-frame 
examination of video data by an analyst versus a real-time flying operation performed by a pilot flying in very low 
visibility conditions. 

This is not the ‘final’ word on EFVS visual advantage, and past performance is not necessarily a reliable indicator for 
future results. In particular, the performance of the EFVS is highly dependent upon the atmospheric conditions, among 
other factors. The visual advantage does demonstrate, however, the significant improvement that the EFVS II system 
installed in the G450 provided during this flight test. 

4. SUMMARY 

A team of Honeywell, Gulfstream Aerospace Corporation, and NASA personnel conducted a flight test with the goal of 
obtaining pilot- in-the-loop test data for flight validation, verification, and demonstration of selected SVS and EFVS 
operational and system-level performance capabilities in actual very low visibility conditions. Nine test flights (38 flight 
hours) were conducted over the summer and fall of 2011. The evaluations were flown in Gulfstream’s G450 flight test 
aircraft outfitted with the certified SVS and EFVS technology. Evaluation pilots flew 108 approaches in low visibility 
weather conditions (as low as 600 ft visibility) into various airports from Louisiana to Maine. 

Video from 73 EFVS approaches were reviewed at decision altitude, 100 ft radar altitude, and touchdown approach 
points to determine if EFVS imagery provided average forward horizontal visible distance increases over that of natural 
vision - a visual advantage. Pilots flew approaches in low visibility weather conditions (600 to 3600 ft reported visibility 
with ceilings between 100 and 500 ft) under different obscurants (mist, fog, drizzle fog, frozen fog) and sky cover 
(broken, overcast). 

Weighted average results indicate the EFVS (FLIR imagery on a HUD) provided increases in average forward horizontal 
visual distance from the cockpit (visual advantage) of two to three times over that of the out-the-window (OTW) view 
for the pilot flying. The EFVS allowed pilots to view the runway environment, specifically runway lights, before they 
would be able to OTW with natural vision. The visual advantage afforded by the EFVS over natural vision provides 
operational data that supports proposed rulemaking where EFVS could be used continuously throughout the approach, 
landing, and roll-out in visibilities as low as 1000 ft RVR in lieu of natural vision. 

The performance of the EFVS sensor was generally outstanding, providing the required visual approach and landing 
references clearly beyond that of the natural vision. Actual weather flying continually demonstrated and emphasized 
how non-homogeneous weather conditions can affect EFVS performance. FLIR-based sensor technology used in 
conjunction with the HUD enabled successful approaches to landings in reported visibility as low as 600 RVR. 
However, EV sensor technologies (other than FLIR) for improved all-weather operations when the reported visibility is 



less than 1000 ft RVR are needed. The visual advantage afforded by the EFVS over natural vision allows a pilot to see 
much more of the approach runway environment (i.e., required visual approach and landing references), increase pilots’ 
situational awareness, and provide more visual cues (e.g., visual momentum, optical flow, etc.) which could aid 
stabilized approach maintenance for a safer landing. 
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